Electronic camera and image processing program

ABSTRACT

An electronic camera includes an imaging unit for continuously capturing images of a subject to create a plurality of frames of RAW data, an evaluation unit for evaluating quality of the RAW data, and a recording unit for selecting a highly evaluated frame from the RAW data according to a result of the quality evaluation by the evaluation unit, and storing the selected frame of RAW data.

TECHNICAL FIELD

The present invention relates to an electronic camera which selects fromcontinuously captured images an image captured in good condition forstorage.

The present invention also relates to an image processing program forcausing a computer to select an image captured in good condition fromimages continuously captured by an electronic camera or the like.

TECHNOLOGICAL BACKGROUND

There is a conventionally known electronic camera that carries out a BSS(Best Captured Selector) operation. The BSS operation is an operation toselect an image that was captured in good condition from a group ofcontinuously captured images and stores the selected image (see JapanesePatent Unexamined Application Publication No. Hei 11-136557, forexample).

On the other hand, there is another known electronic camera that storestherein RAW data (see Japanese Patent Unexamined Application PublicationNo. 2001-223979, for example). RAW data is subjected to almost no imageprocessing, so that it is faithful to a raw output signal of an imagesensor and contains abundant information on an image at a time when theimage has been captured. Moreover, the RAW data generally is large inquantifying bit number and is also a signal containing abundantgradation of the image.

For example, the RAW data mostly has 12-bit gradation in each colorwhile general-purpose image data has 8-bit gradation in each color.

From the above features, the RAW data is suitable for complex orsophisticated image processing.

The conventional electronic camera selects image data containing a largeamount of information on an image according to the file size of a JPEGcompressed file.

For recording the aforementioned RAW data, however, the recording routeor procedure therefor is greatly different from that for typical imagedata, so that no JPEG compressed file is created.

For this reason, no conventional electronic camera has the BSS operationfor RAW data.

DISCLOSURE OF THE INVENTION

It is therefore an object of the present invention to realize a BSSoperation suitable for RAW data.

The present invention will be described below.

(1) An electronic camera of the present invention includes an imagingunit, an evaluation unit, and a recording unit.

The imaging unit continuously captures an image of a subject to generatea plurality of frames of RAW data. The evaluation unit evaluates qualityof the RAW data.

The recording unit selects a highly evaluated frame for storage from theframes of RAW data according to the result of the quality evaluation bythe evaluation unit.

(2) It is preferable that the evaluation unit evaluate quality of imagedata that is generated by interpolating a missing-signal component ofthe RAW data.

In general, in the case where the RAW data in each pixel does notcontain all signal components, a result of the quality evaluation may beinaccurate. However, performing the quality evaluation after theinterpolation of the missing-signal component as described above makesit is possible to evaluate the quality of the RAW data more accurately.

(3) Moreover, it is preferable that the evaluation unit evaluate qualityof the RAW data according to a spatial frequency component of the imagedata whose missing-signal component has been interpolated.

Interpolating the missing-signal component as described above canincrease the number of samples of signal components (sample density),which allows precise detection of the spatial frequency components.Accordingly, it is possible to evaluate quality of the RAW data moreaccurately.

(4) It is preferable that the evaluation unit extract a predeterminedsignal component (hereinafter, referred to as selected signal component)from signal components arranged on a pixel array of the RAW data toevaluate quality of the RAW data according to the selected signalcomponent.

More preferably, the evaluation unit sets a signal component having thesmallest amount of the missing-signal component (e.g., a green componentin a primary color Bayer pattern) as the selected signal component.

The quality evaluation according to only the selected signal componentenables accurate quality evaluation even without the interpolation ofthe missing-signal component. In this case, it is possible to simplifythe procedure of the quality evaluation for the RAW data and perform theBSS operation for the RAW data at high speed.

(5) Moreover, it is preferable that the evaluation unit determine asignal component containing brightness information more than othersignals as the selected signal component to evaluate quality of the RAWdata according to a spatial frequency component of the selected signalcomponent.

Since the selected signal component contains brightness informationmost, it has a higher visual sensitivity than the other signalcomponents particularly. Thus, it is possible to properly select goodRAW data in terms of visual evaluation (e.g., RAW data with visual highsharpness, containing less blurring caused by hand movement).

(6) An image processing program of the present invention is for causinga computer to process a plurality of pieces of RAW data continuouslycaptured by an imaging unit. The characteristics of this imageprocessing program is to cause the computer to function as theevaluation unit and the recording unit as recited in any one of claims 1to 5.

Executing this image processing program makes it possible to select andstore good RAW data by use of the computer.

BRIEF DESCRIPTION OF THE DRAWINGS

The above-mentioned objects and other objects of the invention willbecome easily apparent from the following detailed description when readin conjunction with the accompanying drawing:

FIG. 1 is a block diagram showing the structure of an electronic camera11;

FIG. 2 is a flowchart showing an operation of the electronic camera 11according to a first embodiment of the present invention; and

FIG. 3 is a flowchart showing an operation of the electronic camera 11according to a second embodiment of the present invention.

BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, embodiments of the present invention will be described withreference to the accompanying drawings.

Embodiment 1

FIG. 1 is a block diagram showing the structure of an electronic camera11 according to the present embodiment.

In FIG. 1, a lens 12 is mounted to the electronic camera 11. In an imagespace of the lens 12, a light-receiving plane of an image sensor 13 isarranged. The image sensor 13 is driven by a driving pulse from adriving circuit 13 a and outputs analog RAW data.

The RAW data is input to an A/D converter 15 via an amplifier 14. TheA/D converter 15 converts the RAW data into digital data pixel-by-pixel,thereby obtaining digital RAW data.

The thus obtained RAW data is temporarily recorded in a buffer 16. Then,an image processing unit 17 accesses the buffer 16 and performs colorinterpolation, image compression, or other image processings.

The processing results obtained by the image processing unit 17 isrecorded for storage via a recording interface 18 onto a detachablerecording medium 19.

The electronic camera 11 is provided with a microprocessor 20 forcontrolling the entire system, and an evaluation unit 21. Themicroprocessor 20 controls respective operations of the aforementionedsignal processing system. The evaluation unit 21 extracts an evaluationparameter from data on which the image processings have been performed,for example, to evaluate quality of an image.

Relation Between the Embodiment and the Invention

In the following the relation between this embodiment and the inventionwill be described. It should be noted that the relation is described asa way of example for reference and does not limit the scope of theinvention.

The imaging unit recited in the claims corresponds to the image sensor13, the driving circuit 13 a, and the A/D converter 15.

The evaluation unit recited in the claims corresponds to the evaluationunit 21 and the image processing unit 17.

The recording unit corresponds to the image processing unit 17, therecording interface 18, and the microprocessor 20.

Description on Operations in Embodiment 1

FIG. 2 is a flowchart for explaining the operation of the electroniccamera 11 in the first embodiment. Hereinafter, the operation will bedescribed with reference to step numbers shown in FIG. 2.

Step S1: The microprocessor 20 continuously drives the image sensor 13using the driving circuit 13 a, to capture continuous images of asubject.

By such an imaging operation, the image sensor 13 outputs a plurality offrames of RAW data sequentially. The frames of RAW data are digitalizedby the A/D converter 15 and then recorded in a RAW data region of thebuffer 16 temporarily.

Step S2: The microprocessor 20 instructs the image processing unit 17 toperform a color interpolation processing on the RAW data. The imageprocessing unit 17 accesses the RAW data region and performs colorinterpolation on missing-signal components on the pixel array of the RAWdata in sequence.

Image data is thus created by the color interpolation and temporarilyrecorded in an image data region of the buffer 16.

Step S3: The microprocessor 20 instructs the image processing unit 17 tocompress the image data. The image processing unit 17 accesses the imagedata region and compresses the image data sequentially. The result ofcompressing the image data (compressed data) is temporarily recorded inthe image data region of the buffer 16.

Step S4: The microprocessor 20 instructs the evaluation unit 21 toperform quality evaluation. The evaluation unit 21 accesses the imagedata region and compares the amounts of compressed data for respectivecompression results of the RAW data. The microprocessor 20 then deletesthe compression result with a smaller amount of compressed data and itsoriginal RAW data from the buffer 16 according to the comparison resultby the evaluation unit 21.

By the above operation, the compression result with a large amount ofcompressed data and its original RAW data remain till the end in thebuffer 16. The remaining image data in the buffer 16 is likely tocontain more information on the image and provides a sharp image becauseit has the large amount of compressed data.

Step S5: The microprocessor 20 determines whether or not the electroniccamera 11 is set to a RAW recording mode.

When the electronic camera 11 is set to the RAW recording mode, themicroprocessor 20 advances the operation to Step S6.

On the other hand, when the electronic camera 11 is not set to the RAWrecording mode, the microprocessor 20 advances the operation to Step S7.

Step S6: The microprocessor 20 instructs the recording interface 18 torecord the RAW data. The recording interface 18 selects from the RAWdata recorded in the buffer 16 in Step S1 the RAW data that remains tillthe end in the buffer 16 (having the large amount of compressed data),stores it as a file in the recording medium 19 in the form ofuncompressed data.

A BSS operation for RAW data is completed through a series of operationsdescribed above.

Step S7: The microprocessor 20 instructs the recording interface 18 torecord the compressed image data. The recording interface 18 selects thecompression result of image data remaining till the end in the buffer 16(i.e., the compression result with the large amount of compressed data)from the compression results of image data compressed and recorded inthe buffer 16 in Step S3, and stores it as a file in the recordingmedium 19.

A BSS operation for compressed image data is completed through a seriesof operations described above.

Effects of Embodiment 1

As described above, according to the first embodiment, image data iscreated by interpolating a missing-signal component of RAW data. Then,the quality of the image data is evaluated. Thus, it is less likely thatthe missing-signal component of the RAW data decreases the accuracy ofthe results of the quality evaluation, which improves quality evaluationaccuracy.

Moreover, the above image data is close to a final image that isobtainable by processing the RAW data in exterior, for example, on acomputer. Thus, the quality evaluation is particularly preferable inview of selecting RAW data from which a good final image can be created.

Furthermore, according to the first embodiment, the number of pixels(the number of samples of the image) appears to be increased byinterpolation, so that aliasing and a false color component can besuppressed. Thus, it is possible to improve a situation where thealiasing and false color component makes evaluation of the spatialfrequency components incorrect.

Next, another embodiment of the present invention will be described.

Embodiment 2

The structure of the second embodiment, and relation between theembodiment and the invention are the same as those described in thefirst embodiment, therefore description thereof will be omitted here. Inaddition, from that reason, the same reference numerals shown in FIG. 1are also used in the following description.

Description on Operations in Embodiment 2

FIG. 3 is a flowchart showing the operation of the electronic camera 11in the second embodiment. Hereinafter, the operation will be describedwith reference to the step numbers shown in FIG. 3.

Step S21: The microprocessor 20 determines whether or not the electroniccamera 11 is set to the RAW recording mode.

When the electronic camera 11 is set to the RAW recording mode, themicroprocessor 20 advances the operation to Step S22.

On the other hand, when the electronic camera 11 is not set to the RAWrecording mode, the microprocessor 20 advances the operation to a knownBSS operation.

Step S22: The microprocessor 20 continuously drives the image sensor 13using the driving circuit 13 a, thereby capturing continuous images of asubject.

RAW data output from the image sensor 13 is converted by the A/Dconverter 15 into digital data, and thereafter the digital data istemporarily recorded in the RAW data region of the buffer 16sequentially.

Step S23: The microprocessor 20 instructs the image processing unit 17to extract a selected signal component (a green component in thisexample). The image processing unit 17 accesses the RAW data region andextracts the green component from the RAW data. The extracted greencomponent is temporarily stored in the image data region of the buffer16. For example, in case of RAW data in Bayer pattern, the greencomponent in a checkered pattern is extracted.

Then, the microprocessor 20 instructs the evaluation unit 21 to evaluatequality of the green component. The evaluation unit 21 performs localmultiplicative summation (i.e., so-called spatial-frequency filtering)to the green component to obtain a difference between adjacent pixels.Note that the local multiplicative summation does not include acalculation for a missing part of the green component.

In this manner, only high-frequency components (an edge portion or aregion where gradation largely changes) are extracted from the greencomponent to create an image. The evaluation unit 21 sums absolutevalues of these high-frequency components within a predetermined region(e.g., the entire image plane, the central region of the image plane, ora selected focus detection area), to obtain the amount of thehigh-frequency components.

Step S24: The evaluation unit 21 compares the thus obtained amounts ofthe high-frequency components sequentially. According to the comparisonresult, the microprocessor 20 deletes RAW data having a smaller amountof the high-frequency components from the buffer 16 one by one.

Through the above operation, RAW data having the large amount of thehigh-frequency components remains till the end in the buffer 16. Theremaining RAW data is very likely to contain details of an image inabundance and provides a sharp image because of the large amount ofhigh-frequency components.

Step S25: The microprocessor 20 instructs the image processing unit 17to compress the RAW data. The image processing unit 17 compresses theRAW data remaining till the end in the buffer 16 (the RAW data havingthe large amount of the high-frequency components of the greencomponent).

The recording interface 18 stores the RAW data in the recording medium19 as a file.

Through a series of operations described above the BSS operation for theRAW data is completed.

Effects of Embodiment 2

As described above, in the second embodiment, the green component thatcontains brightness information most is extracted from signal componentson the pixel array of the RAW data to evaluate the amount of thehigh-frequency components for the extracted green component.

As compared with the first embodiment, the color interpolationprocessing on the RAW data is not required in this case. Moreover, it ispossible to simplify the procedure of the quality evaluation because thequality evaluation is needed only for the green component. Consequently,it is easy to substantially increase the speed of the BSS operation forthe RAW data according to the second embodiment.

In addition, the green component has higher visual sensitivity thanother color components, i.e., a red component and a blue component.Thus, by performing the quality evaluation using the spatial frequencycomponents of the green component as a reference, it is possible toaccurately select RAW data that provides a visually sharp image.

Supplemental Items of Embodiments

The aforementioned embodiments have described the electronic camera 11.However, the present invention is not limited thereto. For example, animage processing program may be created by program-coding theoperational processings related to the BSS operation of the presentembodiments.

Moreover, in the aforementioned embodiment, the selected RAW data isrecorded in the form of uncompressed data. However, the presentinvention is not limited thereto. For example, the RAW data may besubjected to lossless compression or lossy compression for recording.The RAW data may be subjected to information amount reduction, trimming,or 90-degree rotation for recording, for example.

In addition, in the aforementioned embodiment the shooting condition isevaluated according to the spatial frequency components. However, thepresent invention is not limited thereto and it can use any evaluationmethod as long as the shooting condition can be evaluated by the method.

For example, an acceleration sensor may be mounted to the electroniccamera so as to measure the amount of hand movement. In this case, RAWdata captured with a small amount of hand movement may be evaluated asRAW data captured in good shooting condition.

Moreover, for example, RAW data in abundant gradation may be evaluatedas the RAW data captured in good shooting condition according togradation (e.g., gradation histogram) of RAW data (or image data createdfrom the RAW data).

Moreover, for example, RAW data in vivid colors may be evaluated as theRAW data captured in good shooting condition according to colorsaturation of RAW data (or image data created from the RAW data).

Moreover, for example, RAW data with a hue that is close to a memorycolor may be evaluated as the RAW data captured in good shootingcondition according to hue of a specific color (e.g., a color of skin ora color of a blue sky) of RAW data (or image data created from the RAWdata).

Moreover, for example, according to hue of RAW data (or image datacreated from the RAW data), RAW data with many different hues may beevaluated as the RAW data captured in good shooting condition.

Moreover, for example, according to a noise in RAW data (or image datacreated from the RAW data), RAW data with a less noise may be evaluatedas the RAW data captured in good shooting condition.

Moreover, for example, RAW data containing edge components mostly in ahorizontal direction or a vertical direction may be evaluated as goodRAW data captured while the electronic camera is in less inclination,according to a degree of inclination of an edge component contained inRAW data (or image data created from the RAW data) within an imageplane.

In the first embodiment described above, image data for qualityevaluation is created by using a full-fledged color interpolationprocessing for image recording. However, the present invention is notlimited thereto. For example, the image data for the quality evaluationmay be created by using another interpolation processing that isdifferent from the full-fledged color interpolation processing (forexample, simple linear interpolation). In this case, it is possible tosimplify the interpolation processing, thereby further heightening thespeed of the BSS operation for the RAW data.

Moreover, in this another interpolation processing it is preferable toprevent the quantifying bit number of the RAW data from decreasing. Inthis case, more precise RAW data quality evaluation is enabled byadditionally evaluating a minute difference in gradation or color tone.

The above-described second embodiment employs the green component as theselected signal component. However, the selected signal component of thepresent invention is not limited to the green component. In general, theselected signal component is preferably a signal component containingbrightness information most among signal components constituting RAWdata.

The invention can be put into practice in various forms withoutdeparting from its spirit or its major aspects. Thus, the aforementionedembodiments merely exemplify the present invention at all points andshould not be interpreted exclusively. The scope of the invention isdescribed by the appended claims but is not bounded by thespecification. Moreover, all changes and modifications within a rangethat is equivalent to the claims fall within the scope of the invention.

INDUSTRIAL AVAILABILITY

An electronic camera of the present invention captures continuous imagesof a subject, to create a plurality of frames of RAW data. Theelectronic camera evaluates shooting conditions for the frames of RAWdata, and selects for storage a highly evaluated frame from these framesof RAW data.

Through such an operation, it is possible to obtain RAW data captured ingood shooting condition with a high probability even if shooting is madeunder a difficult situation.

1. An electronic camera comprising: an imaging unit continuouslycapturing an image of a subject to generate a plurality of frames of RAWdata; a buffer unit temporarily recording the plurality of frames of RAWdata generated by the image unit; an evaluation unit evaluating qualityof the plurality of frames of RAW data recorded in the buffer unit bycomparing information regarding the plurality of frames of RAW data andselecting a highly evaluated frame of RAW data from the plurality offrames of RAW data according to a result of the quality evaluation; anda recording unit recording the highly evaluated frame of RAW dataselected by the evaluation unit into a recording medium.
 2. Theelectronic camera according to claim 1, wherein the evaluation unitevaluates quality of image data that is generated by interpolating amissing-signal component of the RAW data.
 3. The electronic cameraaccording to claim 2, wherein the evaluation unit evaluates quality ofthe RAW data according to a spatial frequency component of the imagedata.
 4. The electronic camera according to claim 1, wherein theevaluation unit extracts a predetermined signal component (hereinafter,referred to as selected signal component) from signal components toevaluate quality of the RAW data according to the selected signalcomponent, the signal components arranged on a pixel array of the RAWdata.
 5. The electronic camera according to claim 4, wherein theevaluation unit determines a signal component containing brightnessinformation more than other signals as the selected signal component toevaluate quality of the RAW data according to a spatial frequencycomponent of the selected signal component.
 6. A computer-readablestorage medium storing an image processing program for causing acomputer to process a plurality of pieces of RAW data continuouslycaptured by an imaging unit, wherein the program causes the computer tofunction as the evaluation unit and the recording unit as recited inclaim
 1. 7. A computer-readable storage medium storing an imageprocessing program for causing a computer to process a plurality ofpieces of RAW data continuously captured by an imaging unit, wherein theprogram causes the computer to function as the evaluation unit and therecording unit as recited in claim
 2. 8. A computer-readable storagemedium storing an image processing program for causing a computer toprocess a plurality of pieces of RAW data continuously captured by animaging unit, wherein the program causes the computer to function as theevaluation unit and the recording unit as recited in claim
 3. 9. Acomputer-readable storage medium storing an image processing program forcausing a computer to process a plurality of pieces of RAW datacontinuously captured by an imaging unit, wherein the program causes thecomputer to function as the evaluation unit and the recording unit asrecited in claim
 4. 10. A computer-readable storage medium storing animage processing program for causing a computer to process a pluralityof pieces of RAW data continuously captured by an imaging unit, whereinthe program causes the computer to function as the evaluation unit andthe recording unit as recited in claim 5.